After running for a few hours, the CPU can still increase to around 50%. It seems to be increasing at a slower pace than v0.5.4, and the "idle wake ups" are smaller.

Another thing I noticed is that different from v0.5.2, the replica keeps generating checkpoints even when no messages are sent. Don't know if this is expected or can cause the cpu usage.

Apr 11 05:30:50.150 INFO Created checkpoint @32758 in 16.030148ms, StateManager: 1

Execution state snapshots are made at an interval to enable purging of older state. But it shouldn't have caused increasing CPU usage. There is something else going on, I'll investigate.

Investigation of the CPU time is tracked here: https://github.com/dfinity-lab/dfinity/pull/3204

A fix to replica is under view at https://github.com/dfinity-lab/dfinity/pull/3270

@ninegua Now that dfinity-lab/dfinity#3270 is merged, can we consider moving this PR forward? This is currently the only blocker on the release (tracking issue #581).

We need to first cut new replica release 0.5.6 before updating the reference revision here. However, there is another problem related to crypto key setup that prevents single node replica from running.

Do we have a notion of cherry pick in the release process, where we can cut a release up to certain PR, plus a set of PRs beyond the cut point?

Do we have a notion of cherry pick in the release process, where we can cut a release up to certain PR, plus a set of PRs beyond the cut point?

That will require a separate branch (e.g. just for v0.5.5) instead of master. I actually have it ready in https://github.com/dfinity-lab/dfinity/pull/3204. But the bigger question is whether it is desirable to maintain a branch release that deviates from master. Let's see what @p-shahi has to say about it.

Do we have a notion of cherry pick in the release process, where we can cut a release up to certain PR, plus a set of PRs beyond the cut point?

That will require a separate branch (e.g. just for v0.5.5) instead of master. I actually have it ready in dfinity-lab/dfinity#3204. But the bigger question is whether it is desirable to maintain a branch release that deviates from master. Let's see what @p-shahi has to say about it.

update: will provide a release from branch with cherry picks for replica 0.5.6
the feature flag discussion will be concluded separately as we need buy in from crypto team on whether or not they want to add such a thing (use-case: feature flag turn off csp key generation and use a set of debug keys)

====
@ninegua it is not desire-able to have a separate branch for just one release with cherry pick fixes but to unblock SDK we can do this.

(if possible, I would suggest we wrap the crypto key setup changes around a feature flag. then, disable this feature for the replica bundled inside dfx - this should be straightforward and wouldn't require a new branch)

@hansl @chenyan-dfinity please communicate the urgency of needing to push out a new SDK release so that I know whether to immediately provide you a new tag (create a new branch with cherry picks) or if we can spend some time trying to get feature flags for this change
(alternatively, you can add this helper script into dfx: https://github.com/dfinity-lab/dfinity/pull/3318/files but is that really what dfx wants to do? @hansl )

superseded by #591